Virtual Reality and Choreographic Practice:The Potential for New Creative Methods

Virtual reality (VR) is becoming an increasingly intriguing space for dancers and choreographers. Choreographers may find new possibility emerging in using virtual reality to create movement and the WhoLoDancE: Whole-Body Interaction Learning for Dance Education project is developing tools to assist in this process. The interdisciplinary team which includes dancers, choreographers, educators, artists, coders, technologists and system architects have collaborated in engaging, discussing, analysing, testing and working with end-users to help with thinking about the issues that emerge in the creation of these tools. The paper sets out to explore the creative potential of VR in the context of WhoLoDancE and how this may offer new insights for the choreographer and dancer. We pay attention to the virtual environment, the virtual performance and the virtual dancer as some of the key components for equipping the choreographer to use in the creating process and to inform the dancing body. The cyclical process of live body to virtual, back to the dancing body as a choreographic device is an innovative way to approach practice. This approach may lead to new insights and innovations in choreographic methods that may extend beyond the project and ultimately take dance performance in a new direction

Investigating Networked Music Performances in Pedagogical Scenarios for the InterMUSIC Project

With the big improvement of digital communication networks, Networked Music Performances (NMP) received a great interest from music live performance and music recording industry. The positive impact of NMP in pedagogical appli- cations, instead, has been only preliminary explored. Within the InterMUSIC project, we aim to investigate NMP from a pedagogical perspective, that has considerable differences with respect to music performances, and to develop tools to improve distance learning experiences. In this paper, we introduce a conceptual framework designed to be the foundation for all the experiments conducted in the project. We also present two preliminary experiments that investigate the sense of presence of geographically-distant musicians in a distance learning scenario. We discuss the comments provided by the musicians as a set of requirements and guidelines for future experiments

Feature-Based Analysis of the Effects of Packet Delay on Networked Musical Interactions

Networked Music Performance (NMP) is a mediated interactional modality with a tremendous potential impact on professional and amateur musicians, as it enables real-time interaction from remote locations. One of the known limiting factors of distributed networked performances is the impact of the unavoidable packet delay and jitter introduced by IP networks, which make it difficult to keep a stable tempo during the performance. This paper investigates the tolerance of remotely interacting musicians towards adverse network conditions. We do so for various musical instruments and music genres, as a function of rhythmic complexity and tempo. In order to conduct this analysis, we implemented a testbed for psycho-acoustic analysis emulating the behavior of a real IP network in terms of variable transmission delay and jitter, and we quantitatively evaluated the impact of such parameters on the trend of the tempo maintained during the performance and on the perceptual quality of the musical interactio

Unsupervised feature learning for bootleg detection using deep learning architectures

The widespread diffusion of portable devices capable of capturing high-quality multimedia data, together with the rapid proliferation of media sharing platforms, has determined an incredible growth of user-generated content available online. Since it is hard to strictly regulate this trend, illegal diffusion of copyrighted material is often likely to occur. This is the case of audio bootlegs, i.e., concerts illegally recorded and redistributed by fans. In this paper, we propose a bootleg detector, with the aim of disambiguating between: i) bootlegs unofficially recorded; ii) live concerts officially published; iii) studio recordings from officially released albums. The proposed method is based on audio feature analysis and machine learning techniques. We exploit a deep learning paradigm to extract highly characterizing features from audio excerpts, and a supervised classifier for detection. The method is validated against a dataset of nearly 500 songs, and results are compared to a state-of-the-art detector. The conducted experiments confirm the capability of deep learning techniques to outperform classic feature extraction approaches

A Dimensional Contextual Semantic Model for music description and retrieval

Several paradigms for high-level music descriptions have been proposed to develop effective system for browsing and retrieving musical content in large repositories. Such paradigms are based on either categorical or dimensional models. The interest in dimensional models has recently grown a great deal, as they define a semantic relation be- tween concepts through graded descriptions. One problem that affects semantic descriptions is the ambiguity that often arises from using the same descriptor in different contexts. In order to overcome this difficulty, it is important to model and address polysemy, which is the property of words to take on different meanings depending on the use-context. In this paper we propose a Dimensional Contextual Semantic Model for defining semantic relations among descriptors in a context-aware fashion. This model is here used for develop- ing a semantic music search engine. In order to evaluate the effectiveness of our model, we compare this engine with two systems that are based on different description models

Three-Dimensional Mapping of High-Level Music Features for Music Browsing

The increased availability of musical content comes with the need of novel paradigms for recommendation, browsing and retrieval from large music libraries. Most music players and streaming services propose a paradigm based on content listing of meta-data information, which provides little insight on the music content. In services with huge catalogs of songs, a more informative paradigm is needed. In this work we propose a framework for music browsing based on the navigation into a three-dimensional (3-D) space, where musical items are placed as a 3-D mapping of their high-level semantic descriptors. We conducted a survey to guide the design of the framework and the implementation choices. We rely on state-of-the-art techniques from Music Information Retrieval to automatically extract the high-level descriptors from a low-level representation of the musical signal. The framework is validated by means of a subjective evaluation from 33 users, who give positive feedbacks and highlight promising future developments especially in virtual reality field

Using multi-dimensional correlation for matching and alignment of MoCap and Video signals

Motion analysis and tracking often relies on multimodal signals, e.g., video, depth map, motion capture (MoCap), due to the completeness of information they jointly provide. The joint analysis of multimodal signals requires to know the correct timing, i.e., the signals to be aligned. In this paper we propose an approach to automatically estimate the correct matching and alignment between a video and a MoCap recording acquired from the same session, based on the multi-dimensional correlation of velocity-based features extracted from the two recordings. We validate our approach over a dataset of dance recordings of four genres, and we achieve promising results for both the alignment and matching scenarios

Time is not on my side : network latency, presence and performance in remote music interaction

A pilot test on the sense of presence and the quality of the interaction in Networked Music Performance is presented.Subjective measures, based on a presence questionnaire, are combined with objective quality metrics, in order tos tress the contribution of temporal factors (i.e., network latency) on the musical experience in the mediated environment. Preliminary results in the scope of chamber music practice are presente

WhoLoDancE: Deliverable 3.5 - Report on data-driven and model-driven analysis methodologies

This deliverable summarizes the description of the development of techniques adopted for multimodal analysis of dance at both individual and group levels, data-driven, and model-driven analysis. Section 1 introduces the report and lists its objectives whereas Section 2 refers to the methodology employed in the data-driven approach. Section 3 provides an overview of developed model-driven approaches to extract movement dimensions related to the dance-learning scenario: from low-level model-based movement dimension to more complex intra- and inter- network related methodologies, including a technique to automatically segment dance sequences in meaningful chunks